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1.  Introduction 

This  study  originated  in  an  attempt  to  understand  the  meaning  of  two  pa¬ 
rameters,  the  mean  and  the  variance,  which  appear  in  the  limit  theorems  con¬ 
cerning  sums  of  identically  distributed  independent  random  variables.  The  law 
of  large  numbers,  when  written  in  the  usual  form  (lim  Sn/n  =  /x)  does  not  im¬ 
mediately  suggest  a  natural  generalization  to  Markov  chains;  the  reason  is  that 
a  Markov  chain  takes  its  values  in  an  abstract  state  space  so  that  not  only  the 
limit  n,  but  also  the  ratios  which  tend  to  n  have  to  be  reinterpreted  in  a  meaning- 
,  ful  fashion.  Therefore,  we  proceed  to  the  renewal  theorem  in  the  form  proved  by 
Erdos,  Feller,  and  Pollard  (cf.  [3],  p.  286)  and  Chung  and  Wolfowitz  [2],  and 
formulate  it  in  a  manner  which  readily  suggests  a  natural  generalization. 

Consider  a  random  walk  (spatially  homogeneous  Markov  chain)  Xn  on  the 
integers  with  transition  function  P{x,  y)  defined  for  arbitrary  pairs  of  integers 
X,  y.  Suppose  that  it  satisfies 

(1.1)  P(^,y)>0,  i,  P(x,y)  =  l, 

5/=  — » 

(1.2)  P(x,  y)  =  P(x  z,y  +  z)  for  all  z, 

(1.3)  E  |a:|  P(0,  x)  <  <=0,  E  xP(0,  x)  =  y  >  0, 

X  =  —  00  —  00 

(1.4)  E  and  |/(a;)|  <  l=>f(x)  =  constant. 

y=—to 

The  last  condition  (1.4)  is  well  known  to  be  equivalent  to  the  usual  aperiodicity 
requirement  that  the  support  of  P(0,  x)  is  not  contained  in  a  proper  subgroup 
of  the  integers  (cf.  [4],  p.  276).  This  Markov  chain  is  transient  in  view  of  con¬ 
dition  (1.3),  ([4],  p.  33),  and  the  renewal  theorem  is  a  simple  statement  concern¬ 
ing  the  asymptotic  behavior  of  the  Green  function  G{x,  y)  defined  by 

G{x,  y)  =  't  Pn{x,  y), 

n  =  0 

(1.5)  Po(x,  y)  =  8{x,  y), 

Pn+iix,  ?/)  =  E  Pn{Xy  z)P{z,  y), 

Zsa  —00 
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The  renewal  theorem  states  that  conditions  (1.1)  through  (1.4)  imply 

(1.6)  lim  G{x,  y)  =  -• 

The  mean  n  measures,  in  some  sense,  the  rate  of  drift  of  the  Markov  chain  to  its 
boundary.  To  see  this  phenomenon  in  its  simplest  setting,  suppose  in  addition 
to  (1.1)-(1.4)  that  the  random  walk  can  only  go  to  the  right,  namely  that 

(1.7)  P{x,  y)  =  0  when  y  <  x, 
and  let 

(1.8)  Tn  =  min  [A:]  Xk  >  n\. 

Then  a  weak  (Cesaro)  version  of  (1.6)  may  be  written  in  the  form 

(1.9)  E  (?(0,  x)  =  EolTn]  ~  -  as  n  00 . 

x=0  M 

Consider  now  a  Markov  chain  Xn  w'hose  state  space  consists  of  the  integers, 
and  whose  transition  function  P{x,  y)  satisfies  (1.1)  and  (1.7),  but  neither  (1.2), 
(1.3),  nor  (1.4).  Its  Green  function  G{x,  y),  defined  by  (1.5),  will  obviously  be 
finite,  and  the  identity  in  (1.9)  is  still  correct.  The  problem  of  understanding  the 
role  of  /X  may  therefore  be  phrased  as  follows :  to  find  a  function  /,  defined  in  a 
natural  manner,  so  that 

(1.10)  E  <j(0,  ^)  = -E'o[T„]  ^/(w)  as  n-^oo. 

x  =  0 

We  shall  proceed  to  show  that  this  problem  may  be  solved,  for  a  large  class  of 
Markov  chains  Xn,  with  countable  state  space  S,  by  choosing  for  /  a  solution  of 
the  equation 

(1.11)  E  P(x,  y)f(y)  -  f(x)  =  1,  E  P(x,  y)\f(y)\  <  «>,  x  e  S. 

vES  VES 

(We  shall  simply  write  Pf  ~  f  =  1,  in  the  sequel,  to  indicate  that/ satisfies  the 
equation  in  (1.11)  and  is  summable  with  respect  to  P.)  In  the  case  of  random 
walk  satisfying  (1.1),  (1.2),  and  (1.3),  the  equation  Pf  —  f  =  1  is  indeed  satisfied 
by  the  function 

(1.12)  /(n)  =2.  re  =  0,  dzl,  ±2,  •  •  ■  , 

as  an  elementary  calculation  shows.  It  is  perhaps  more  remarkable  that  even  the 
variance  of  random  walk  is  closely  related  to  equation  (1.11).  For  suppose  that 
(1.1)  and  (1.2)  hold,  but  that  (1.3)  is  replaced  by 

(1.13)  E  xP{Q,  x)  =  =  0,  0  <  0-2  =  E  x^P{0,  a:)  <  00. 

a:=  —  00 

Then  (1.12)  is  meaningless,  while 

(1.14)  /(re)  = 
now  becomes  a  solution  of  Pf  ~  f  =  1. 


71  =  0,  ±1,  ±2,  •  •  • 
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In  section  2  we  show  how  certain  solutions  of  (1.11)  lead  to  a  law  of  large 
numbers  for  Markov  chains  which  generalizes  the  usual  law  of  large  numbers. 
In  section  3  we  give  sufficient  conditions  for  the  weak  renewal  theorem  (1.10)  to 
hold.  Finally,  section  4  is  devoted  to  a  renewal  theorem  for  a  class  of  Markov 
chains  which  generalizes  the  renewal  theorem  (1.6)  for  random  walk  with  finite 
second  moment.  It  is  the  only  satisfactory  result  of  this  paper,  in  the  sense  that 
it  takes  the  form  of  a  uniqueness  theorem  for  solutions  of  equation  (1.11)  satis¬ 
fying  certain  growth  conditions.  And  this  is,  of  course,  the  heart  of  the  matter; 
we  have  to  learn  how  to  single  out  that  solution  of  (1.11)  which  is  the  natural 
analogue  of  the  linear  function  in  (1.12),  and  this  causes  grave  difficulties. 

It  is  easy  to  construct  Markov  chains  for  which  (1.11)  has  no  solution  at  all. 
On  the  other  hand,  there  may  obviously  be  many  solutions,  since  f  +  h  satisfies 
(1.11)  whenever/  does  and  h  is  harmonic,  that  is,  satisfies  Ph  =  h.  But  while 
boundary  theory  characterizes  the  nonnegative  harmonic  functions,  nothing  is 
known  about  the  set  of  all  harmonic  functions. 

Finally,  it  may  even  happen  that  (1.11)  has  one,  or  even  more  than  one,  solu¬ 
tion,  but  that  none  of  them  are  sufficiently  well  behaved  to  yield  even  a  weak 
limit  theorem  of  the  type  of  (1.9). 

2.  A  law  of  large  numbers 

Let  us  assume  that  Xn  is  a  Markov  chain,  whose  state  space  is  a  countable  set 
S,  and  whose  transition  function  P  satisfies  (1.1)  (with  the  summation  over  S, 
of  course).  A  martingale  convergence  theorem  due  to  Chow,  Mallows,  and 
Robbins  [1]  states  that  if  is  a  centered  sequence  of  random  variables  (the 
conditional  expectation  of  ^n+i  given  ^i,  •  •  •  ,  is  0)  and  if  2]  <  0°  for 

some  1  <  a  <  2,  then  the  series  Z)  converges  with  probability  one.  As  an 
immediate  consequence,  we  can  obtain  a  strong  law  of  large  numbers  for  Xn. 

Theorem  2.1.  Suppose  that  the  transition  function  P  of  Xn  is  such  that  the 
equation  Pf  —  f  =  1  has  a  solution  f  uith  the  property  that 

(2.1)  L  P{x,  y)\f{y)  —  fix)  -  1(“  <  M,  for  all  x  e  S, 

yES 

for  some  a  >  1  and  some  M  <  ^.  Then 

(2.2)  P,  r  lim  =  ll  =  1  for  each  x  e  S. 

Ln-^oo  n  J 

Proof.  We  write 

(2.3)  f  (xn)  ~  f  (xo)  ~  n 

=  E  yk,  yk  =  fixk+i)  —  f(xk)  -  1,  k  >0,  n>  1, 

k  =  0 

and  observe  that  in  view  of  Pf  —  J  =  1,  the  sequence  /  (x„)  —  n,  for  w  >  0,  is  a 
martingale  with  respect  to  the  fields  =  {^o,  xi,  •  •  •  ,  x„} ,  w  >  0. 
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For  the  same  reason,  the  sequence 

(2.4)  2.  =  ”Z  n  >  1 

A:  =  l  k 

is  also  a  martingale  with  respect  to  the  same  fields.  By  use  of  Kronecker’s  lemma, 
one  can  obtain  (2.2)  if  one  knows  that  the  limit  of  exists  and  is  finite  with 
probability  one  with  respect  to  the  measure  Pa:[-]  for  each  x  (in  other  words,  for 
each  starting  point  Xa  =  x  oi  the  Markov  chain).  But  in  view  of  the  theorem  of 
Chow,  Mallows,  and  Robbins,  the  series  in  (2.4)  will  converge  with  probability 
one  for  each  x  if 

(2.5)  f  E. 

k=0 

Now  (2.1)  implies  that 

(2.6)  Ex\yk\“  =  E:,\f(xk+i)  —  f{xk)  -  1|“ 

=  Z  Px[xk  =  y]ll  P(y,  z)\fiz)  -  f(y)  -  1|“  <  M, 

VES  ZES 

so  that  (2.5)  holds,  and  the  theorem  is  proved. 

There  are  examples  which  show  that  the  conclusion  of  the  theorem  is  false 
when  (2.1)  only  holds  with  a  =  1.  An  interesting  open  question  concerns  the 
Green  function  of  a  Markov  chain  satisfying  the  hypotheses  of  the  theorem.  It 
follows  from  (2.2)  that  the  process  must  be  transient  so  that  G(x,  y)  <  <*>  for  all 
x  and  y  in  S,  but  it  is  not  clear  whether  the  existence  of  a  function  /  satisfying 
(2.1)  implies  that  G{x,  x)  must  be  bounded. 

It  is,  of  course,  possible  to  rephrase  the  theorem,  by  making  sufficiently  strong 
assumptions  concerning  the  Green  function  of  the  process  Xn,  so  that  the  hy¬ 
potheses  of  the  theorem  become  conclusions. 

Theorem  2.2.  Suppose  that  the  Markov  chain  is  transient,  and  that  its  Green 
function  has  the  property  that 

(2.7)  h{x,  y)  =  Y.  \G(x,  z)  -  G{y,  z)\  <^,  x,y  eS, 

zES 

and  that 

(2.8)  Z  P(x,  y)\h(x,  y)|“  <  M,  x  e  S,  a  >  1. 

VES 

Then,  for  any  fixed  point  0  E  S, 

(2.9)  fix)  =  Z  [G(0,  z)  -  Gix,  z)] 

zES 

is  a  solution  of  Pf  —  f  =  1,  and  the  strong  law  (2.2)  holds. 

Proof.  It  follows  from  (2.8)  that  the  function  /  defined  by  (2.9)  is  summable 
with  respect  to  P.  Also  (2.7),  together  with  (2.8),  yields  the  dominated  conver¬ 
gence  necessary  to  interchange  limits  in 

(2.10)  Pfix)  -  fix)  =  z  Z  Pix,  y)[GiO,  z)  -  Giy,  z)]  -  fix) 

zES  yES 


<  w 


for  each  x  E  S, 
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Now  (2.8)  and  (2.9)  imply  (2.1),  so  that  the  strong  law  follows  from  the  previous 
theorem. 


3.  A  weak  renewal  theorem 


Let  us  investigate  conditions  on  a  Markov  chain  Xn,  under  which  the  stopping 
times  Tn  defined  in  (1.8)  have  the  property  that 


(3.1) 


lim 

n— >00 


En[Tn\ 

fin) 


=  1 


for  some  solution  /  of  the  equation  Pf  —  f  =  1.  For  (3.1)  to  make  sense  it  is 
necessary  that  the  state  space  S  be  ordered.  Therefore,  we  may  take  for  S  the 
positive  integers,  but  this  assumption  is  of  course  quite  vacuous  unless  we  assume 
that  the  transition  function  P  is  somehow  well  behaved  with  respect  to  this 
particular  ordering.  This  motivates  the  following  hypotheses  {A)  (which  can  be 
weakened  in  some  respects,  but  at  the  cost  of  complicating  the  analysis  without 
leading  to  really  satisfactory  results). 

(A)  The  Markov  chain  Xy,  >  0,  has  as  its  state  space  the  positive  integers. 
Its  transition  function  P{k,  /)has  the  boundedness  property  that  the  chain  can 
take  only  a  finite  number  of  steps  to  left  or  right :  there  is  an  Af  >  0  such  that 
P{k,  t)  =  0  when  |^  —  A:|  >  M.  Finally,  it  is  assumed  that  the  chain  has  no  fi¬ 
nite  closed  set  of  states:  there  is  no  finite  subset  C  of  the  integers  such  that 
^tGcP(k,  ^)  =  1  for  all  k  e  C. 

Note  that  hypothesis  (A)  does  not  imply  that  the  chain  x,  is  transient.  Thus 
the  Green  function  may  be  infinite,  but  the  stopping  times  T„  in  (3.1)  are  finite 
with  probability  one  and  have  finite  expectation.  (This  follows  from  our  assump¬ 
tion  that  there  are  no  closed  sets;  therefore,  the  first  time  Tn  at  which  the  chain 
Xy  is  outside  the  set  [1,  2,  •  •  •  ,  n\  has  the  property  that  the  tails  P*[T„  >  m] 
decrease  at  least  exponentially  as  m  for  each  fixed  x  and  n.) 

Theorem  3.1.  Suppose  that  the  Markov  chain  Xy,  r  >  0,  satisfies  {A),  and  let 
Tn  =  min  [v\v  >  0,  x,  >  n].  Suppose  further  that  Pfi  —  f  =  I  has  a  solution  f 
such  that  f(n+  l)/f  (n)  1  as  n-^  -f-oo .  Then 

(3.2)  lim  =  1,  for  each  A:  >  1. 

n^oo  J  \Jl) 

Proof.  We  begin  by  showing  that  any  solution  of  Pf  —  f  =  1  such  that 
f  (n  +  l)//(n)  — >  1  has  the  property 

(3.3)  lim/(n)  =  -f=o. 


Suppose  that  /  is  such  a  solution,  and  let 

(3.4)  m„  =  max  f{k). 

1  <k<n 

It  is  impossible  that  /  (k)  =  w„  when  k  <  n  —  M,  since  k  <  n  —  M  implies 

k->rM 

(3.5)  f{k)  =  Z  P{k,  e)f{e)  -  1  <  -  1. 

t=k-M 
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Therefore,  /  assumes  its  maximum  in  [1,  2,  •  •  •  ,n]  at  one  of  the  points 
of  the  interval  [w  —  ilf  +  1,  n].  But  then  the  identity  in  (3.5)  implies  that 
f{k)  >  m„  +  1  at  one  of  the  points  k  in  the  interval  [n  +  1,  w  +  M].  Now  we 
may  repeat  the  above  argument,  with  n  replaced  hy  n  M  and  obtain  the 
following  conclusion:  there  exists  an  increasing  sequence  of  integers  Ui  <  < 

Uz  <  •  •  •  such  that 

(3.6)  Wfc+i  —  nk<  M  and  /(Wit+i)  -  f{nk)  >  1. 

Thus/(nA:)  ^  +00  as  A;  00,  and  (3.6),  in  conjunction  with  the  hypothesis  that 
f{n  +  l)//(n)  1,  implies  (3.3). 

The  rest  of  the  proof  depends  on  the  identity 

(3.7)  EkUixTs)]  —  Ek[Tn\  =  fik),  k>l,n>l. 

(It  should  be  clear  how  (3.7)  implies  the  theorem;  simply  divide  (3.7)  by/ (n)  and 
let  n  00 ,  observing  that  / {xr^/f  {n)  — >  1  boundedly  with  probability  one,  since 
\xTi,  —  n\  <  M.) 

Equation  (3.7)  is  suggested  by  the  fact,  observed  in  the  proof  of  theorem  3.1, 
that  f{xn)  —  n  is  a  martingale.  Thus  for  every  bounded  stopping  time  T, 
EkUi^r)  —  T]  =  f(k).  But  since  the  stopping  times  Tn  are  not  bounded,  we 
have  to  use  the  slightly  stronger  Markov  property  instead. 

Let  C  be  an  arbitrary  finite  set  of  positive  integers  and  T  =  min  \y\v  > 
0,  Xv  G  where  V  is  the  complement  of  C.  For  each  integer  x, 

m 

(3.8)  E.UixT)]  =  lim  E  r  P.[x,  =  z;T>y]  E  P(^,  y)f(y) 

m— >00  y=0  zGC 
m 

=  lim  E  L  P.[x,  =  z;T>  p]{f(z)  +  1  -  E  P(^,  y)m} 

7^— >00  yz=o  zQC  y^C 

£  E  Px[xy  =  z;T  >  v][f{z)  +  1] 

V  =  0 

m+\  ') 

-EE  Px[xy  =  z;T  >  p]f{z)  k 
y=i  zee  J 

=  lim  /e!  Px[T  >  v]  +f{x)  ~  E  Pxlxy  =  z)T  >  m  +  1]/(2)V 

Ll'  =  0  Z&J  J 

Hence, 

(3.9)  -  E,[T]  -  f(x)  =  lim  E  ■?.[*.  =  i;T>m  + 

TO— z£C 

The  right-hand  limit  is  0  since,  if  |C|  denotes  the  cardinality  of  C, 

(3.10)  I  E  Px[xy  =  z,T  >  m  -h  1]/(2:)|  <  |C|  max  \f{x)\Px[T  >  m  +  1]  — >0. 

z  X  GC 

Thus  (3.7)  is  proved  by  setting  T  =  Tn,  and  that  completes  the  proof  of  the 
theorem. 

Just  as  was  done  in  section  2,  one  can  modify  theorem  3.1  so  that  the  hy¬ 
potheses  concerning  /  become  conclusions  instead.  Suppose  that  a  process  Xy 
satisfies  (A)  and  that  in  addition  G{x,y)  <  <»,  and 


(3.11) 
and 

(3.12) 
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00 


fix)  =  E  [G(l,y)-G(x,y)]  < 

y=l 


for  each  x, 


lim 

as— »+«> 


/(^  +  1) 
fix) 


=  1. 


Then  it  follows  that  the  function/ defined  by  (3.11)  satisfies  Pf  —  f  =  1.  There¬ 
fore,  the  conclusion  of  theorem  3.1  is  valid  and  takes  on  the  curious  form 


(3.13) 


lim 

n— >00 


1 

E,[T.] 


00 


E 

y  =  l 


[Gil,  y)  ~  Gin,  y)] 


=  lim 

n— >00 


1 

Eu[Tn] 


lim  {E,[Tm\  -  =  1, 


k  >  1. 


As  an  illustration  of  theorem  3.1  in  the  recurrent  case,  consider  a  random  walk 
on  the  integers  (positive  and  negative)  satisfying  (1.1),  (1.2),  and  (1.3)  with 
mean  /n  =  0.  Assume  also  that  the  possible  displacements  are  bounded.  Then 
0-2  <  00 ,  and  we  shall  be  concerned  with  the  solution  f  oi  Pf  —  f  =  1,  given  in 
equation  (1.13).  It  is  natural  to  order  the  state  space  in  such  a  way  that/becomes 
monotone.  Therefore,  we  map  the  integers  on  the  positive  integers  according  to 
the  formula  k—^2k  for  k  >  0  and  k—^2\k\  —  1  when  A;  <  0.  Thus  the  random 
walk  induces  a  Markov  chain  on  [1,  2,  3,  •  •  •)  for  which  Pf  —  f  =  1  has  the 
solution 


(3.14) 


/4<r2 

+  l)V4(r2 


for  even  /  >  1, 
for  odd  /  >  1* 


The  conditions  of  theorem  3.1  are  satisfied,  becomes  the  time  elapsed  until 
the  random  walk  on  the  group  of  all  integers  leaves  an  interval  which  is  approxi¬ 
mately  [—n/2,  n/2],  and  theorem  3.1  yields  the  well-known  conclusion  Ek[Tn} 
n as  n  — »  00 ,  for  each  fixed  k. 


4.  A  renewal  theorem 

Here  we  make  drastically  stronger  assumptions  than  in  section  3,  in  order  to 
arrive  at  a  generalization  of  the  classical  renewal  theorem  for  positive  random 
variables.  We  consider  processes  Xy,  v  >0,  satisfying  conditions 

(B)  (i)  Xv,v'>Q  is  a  Markov  chain  on  the  'positive  integers,  which  can  move 

only  to  the  right;  that  is,  Pik,  ^)  =  0  when  t  <  k; 

(ii)  the  chain  has  uniformly  integrable  second  moments,  namely,  the  tails 
are  assumed  uniformly  hounded  so  that 

(4.1)  r(w)  =  sup  E  Pik,j)<^; 

y  =  n+fc+l 

and  in  addition  nr(n)  <  oo  ; 

(iii)  the  only  hounded  harmonic  functions  are  the  constants,  that  is,  Pf  =  /, 
and  1/1  <  M  =^f  =  constant. 
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The  renewal  theorem  takes  the  form 

Theorem  4.1.  If  a  Markov  chain  satisfies  (B)  and  if  the  equation  Pf  —  f  =  1 
has  a  solution  f  whose  increments  are  hounded  (|/(n  4-  1)  —  /(n)|  <  ilf  for  al^ 
n  >  1),  then 

(4.2)  f{n)  -  f(k)  =  f:  lG(k,  I)  -  G(n,  I)]  for  all  k  >  1,  n  >  1. 

<  =  i 

Before  proceeding  to  the  proof,  let  us  observe  how  this  uniqueness  theorem  for 
a  class  of  solutions  of  Pf  —  f  =  1  reduces  to  the  ordinary  renewal  theorem  for 
random  walk. 

Corollary  4.1.  Let  P(k,  t)  be  the  transition  function  of  a  positive  aperiodic 
random  walk  on  the  positi  ve  integers  with  finite  second  moment,  that  is,  assume  that 

(4.3)  P{k,  1)>Q,  t  P(k,  ^)  =  1,  f  I^Pik,  t)<^, 

1=1  t=i 

P{k,  I)  =  P(l,  t-k  +  l), 

and  that  the  greatest  common  divisor  of  [A:(P(1,  1  +  A:)  >  Q]  is  1.  Let  G(k,  ()  he  the 
Green  function  defined  in  (1.5).  Then 

(4.4)  lim  G{1,  w)  =  -  where  m  hP{l,  1  +  k). 

n— >00  U  k  =  l 

Proof  of  the  corollary.  All  the  hypotheses  for  theorem  4. 1  are  clearly 
satisfied  if  we  choose /(n)  =  n/p,  n  >  1.  Now  set  w  =  2  and  A:  =  1  in  equation 
(4.2).  It  reduces  to 

1  m 

(4.5)  i  =  lim  E  [G{1,  t)  -  G{2, 1)] 

fi  fn~* t— I 
m 

=  lim  Z  [G(l,  I)  -  G(l,  !-!)]=  lim  G(l,  m). 

m— »«  t=l  m—Ko 

Proof  of  the  theorem.  First  we  need  the  fact  that 

(4.6)  lim  lG(k,  n)  —  G{I,  n)]  =  0 

n— 

for  every  pair  of  positive  integers  k  and  1.  This  is  a  standard  ingredient  of  every 
potential  theoretical  proof  of  the  renewal  theorem.  If  (4.6)  were  false  for  some 
pair  k,  I,  with  k  I,  then  there  would  exist  a  subsequence  n'  of  the  positive 
integers  such  that  the  limit  in  (4.6)  along  this  subsequence  is  a  0.  (Here  and 
in  the  sequel  we  use  the  fact  that  0  <  G(k,  /)  <  1,  in  view  of  condition  B(i).) 
Taking  a  further  subsequence  n"  of  n'  with  the  property  that  lim„"->«  G{k,  n")  = 
<t>{k)  exists  for  all  A:  >  1,  we  find  that  namely  that  is  a  bounded  har¬ 

monic  function.  However,  <f>{k)  —  4>{l)  =  a  9^  0,  which  shows  that  <{>  is  not 
constant.  That  contradicts  assumption  B(iii)  and  therefore  proves  (4.6). 

Now  define,  as  usual  Tm  =  min  [v\v  >  0,  >  w],  observe  that  in  view  of 

B(i), 

m 

(4.7)  'L^Gik,!)  =  Ek[Tml 
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and  that 

(4.8)  E,U{xt:)]  =  k>l,  m  >  1. 

(The  proof  of  (4.8)  as  given  in  section  3  depends  on  B(i);  neither  B(ii)  nor  B(iii) 
are  used.)  Combining  (4.7)  and  (4.8),  we  obtain 

m 

(4.9)  E  [G(l,  -  G(n,  ()]  -  [/(«)  -  /(!)] 

<=1 

=  Ei[f(XT„)]  —  En[f{XT„)] 

=  Ei[f{xT„)  -  f{m)]  -  EnUixT,^  -  /(w)]. 
Decomposing  these  expectations  according  to  the  place  of  the  last  visit  of  the 
process  to  the  interval  [1,  m],  one  obtains 

m 

(4.10)  E  [G(l,  1)  -  G{n,  t)]  -  U{n)  -  /(I)] 

<=i 


=  E  [G{l,j)  -  G{n,j)]  E  P{j,  m  +  k)[f(m  +  k)  -  /(m)]. 

j  =  l  ib  =  l 


Now  set  lG(l,j)  —  G(n,j)l  =  e,  and  observe  that 
(4.11)  |/(m  +  A:)  —  /(w)|  <  kM, 

Then  (4.10)  gives 


k>  1. 


(4.12)  E  [G(l,  i)  -  G{n,  1)]  -  [/(n)  -  /(I)] 
^=1 


By  condition  B(ii), 


m  00 

<  ^  E  ey  E  kP{j,  m  +  k). 

y  =  i 


(4.13)  E  kP{j,  m-\-  k) 

k  =  l 


=  E  E  PUJ  +  j  +  r)  <  E  r{m  -  j  -j-  k  -  1), 

k=l  r=k  k=l 


SO  that 


(4.14)  M  €j  kP{j,  m  k)  <  M  r{m  ~  j  +  k) 

y=ift=i  y=iA=o 


=  ilf  E  Cm-ySy,  where  6y  =  E  r(/c). 

y=i  k=j 


By  B(ii)  E”  s-nd  by  (4.6),  ey— >^0.  Therefore, 


(4.15) 


lim  E  Cm-y5y  =  0, 
TO— y  =  i 


which  implies,  according  to  (4.12)  and  (4.13),  that 


(4.16) 


lim  E  [<^(1,  ^)  -  G{n,  t)]  =  f(n)  ~  /(I). 

TO— (=l 
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Thus  we  have  proved  (4.2)  in  the  case  when  A:  =  1,  but  this  special  case  of  (4.2) 
is  easily  seen  to  imply  the  general  case.  That  completes  the  proof. 

Many  open  problems  remain,  however.  If  one  tries  to  dispense  with,  or  weaken, 
condition  B(i),  then  it  is  hard  to  derive  (4.6)  as  the  Green  function  G(k,  1)  need 
apparently  no  longer  be  bounded.  Keeping  B(i)  and  B(iii),  however,  it  seems 
certain  that  theorem  4.1  should  hold  under  a  weaker  version  of  B(ii).  It  is  not 
known  whether  it  suffices  to  assume  that  the  tails  are  uniformly  summable, 
namely  that  Z]  r(n)  < 
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